parser: refuse a modifier-rescue and a not/nested-! command in an arg position, and require every clause terminator - #38

What this refuses, and why the gate is a table and not a corpus

v0.6.0 taught the parser that an arg_value position has no command
alternative. Five shapes of that family survived, reported against v0.6.0. The
bibliography turned them into four productions, one of which is not about
command at all, and the enumerated table grew from 76 shapes to 319.

The productions, mapped to our functions

Read from parse.y at ruby/ruby tag v3_4_0. Line numbers are that file's.

Four findings, and two of them contradict the brief I was given

1. There is no arg: arg modifier_rescue arg production. Line 4162 belongs to
arg_rhs, not arg. A modifier-rescue lives in exactly two families:
stmt (3184) and the assignment right-hand sides (arg_rhs 4162,
command_rhs 3323, mrhs_arg 3219, endless_arg 4081). So f(x rescue nil)
is not "a separate rule" from the arg_value family — it is the same rule,
and one flag expresses both. The carve-out is what makes the picture falsifiable:
[a = x rescue nil], f(a = 1 rescue nil), f(a.b = 1 rescue nil),
f(a ||= 1 rescue nil) and a[a = 1 rescue nil] are all legal MRI in the
very positions that refuse the bare form.

2. exc_list already had permitCommand(false); the defect was elsewhere.
parseSplatOrExpr has carried that call since v0.6.0, and it worked:
commandArgsFollow did decline to give foo the argument bar in
rescue foo bar. The leftover bar was then read as the clause's first
statement, and the clause parsed. opt_rescue's then — term | keyword_then | term keyword_then — is not optional, and neither is case_body's, nor
k_if's, nor expr_value_do's. begin; x; rescue Foo end, case x; when 1 end,
if x end, while x end and for i in a end are all SyntaxErrors in MRI, and we
accepted every one of them. A refused production only surfaces as an error when
something downstream insists on what must come next.

3. The ! asymmetry is real and decides where the fix goes. !foo bar is
legal, !!foo bar is not, because expr: '!' command_call takes a
command_call — not an expr. A command_call is never itself a unary
expression, so the privilege cannot nest. The fix belongs at the operator
(one level of ! consumes it), not at the operand.

4. defined? unparenthesised is an arg production, and v0.6.0 refused it.
arg: keyword_defined '\n'? begin_defined arg (4021) is an arg, not a
command, so it is available in an arg position: [defined? x],
{k: defined? x}, 1 + defined? x, f(1, defined? x) and
def m(a = defined? x); end are legal MRI. v0.6.0 gated it behind
commandArgsFollow, whose first act is to refuse everything in an arg
position — a second regression of the same shape as the [defined?(foo bar)]
one v0.6.0 caught before shipping. Seven table shapes and two real stdlib
files were refused because of it.

One mechanism, extended — not a second one

rescueModOK replaces noRescueMod with the sense inverted, so the zero value
is the refusing one and only the positions MRI gives a production to turn it
on. permitRescueMod is its only writer and withRescueModifier its only
reader, exactly mirroring argOnly / permitCommand / commandArgsFollow. The
!, not and defined? fixes are all argOnly; they add no flag.

Gate: the enumerated table

319 shapes, each judged on ruby 4.0.5 by both ruby -c and
ruby --parser=parse.y -c. Any shape whose legality is contextual is written
inside the context that makes it legal, so no -c verdict here is a semantic
refusal dressed as a grammar one.

A/B: the new table, dropped unchanged onto a clean origin/main checkout in a
separate worktree, fails 122 = 115 + 7 assertions and compiles.

The 13 disagreements are all not in a position that admits an expr: prism
accepts x = not y, f(not x), a[not x], p(not x), a.b(not x),
x ||= not y, return not x, yield not x, foo not x, p not x, !not x,
!not foo bar; parse.y refuses them. We keep prism's answer and assert
nothing. (a[foo bar do end] and f(foo bar do end), the two already known, are
likewise absent from both tables.)

The tables in arg_position_command_test.go are now 219 refused and 259
accepted shapes, extended in place.

Floor: two corpus sweeps, with their populations named

The sweeper is its own integrity guard and exits non-zero on any of: a row with
an empty path column, distinct paths ≠ input count, or fewer than
-min-ok files parsing — the floor being the baseline's own OK count, so a run
that read nothing cannot pass. All six runs printed INTEGRITY OK.

The two stdlib files that moved the other way are ripper/filter.rb and
ripper/lexer.rb, and they are the nicest result here: both use

if m = token_match(src, pattern)
then m.string(n)
else nil
end

— MRI's third spelling of then, term keyword_then. v0.6.0 tried
p.accept(token.THEN) before skipping the newline and left the then as a stray
statement. Implementing the clause terminator as the required nonterminal it
is refused 115 shapes and accepted two real files at the same time.

Controls, all still holding (re-measured against both MRI parsers)

• while begin; y do end; end; end — still a SyntaxError; so are while y do end; end and while x = y do end; end.
• foo :a, (y do end) and x = (y do end) — still accepted.
• The eight loop-condition shapes from A do...end block in a loop condition is still refused: the COND half of #32 #33/parser: clear the COND bit-stack on entering a grouping construct, so a do…end block parses in a loop condition #34 — while (y do end); end, while [y do end]; end, while a[y do end]; end, while {k: y do end}; end, until [y do end]; end, foo :a, [y do end], foo :a, {k: y do end}, foo :a, (y do end) — all still accepted.
• a[foo bar], a[foo bar] = 1, a[foo bar] += 1, f(foo bar, 1), a[foo bar, 1] — still accepted. The rule stays head-only.
• [defined?(foo bar)] — still accepted.
• raise "x" rescue 42 still parses as (raise "x") rescue 42, asserted on the AST, not on acceptance.

Coverage, four parts

1. Not compared as rounded totals: the below-100 function list is diffed.
2. That list is identical — 64 entries, every one an ast/ast.go node()
   marker method. No parser.go function is below 100%.
3. Each go tool cover -func was resolved in the tree that produced its
   profile (origin/main in its own worktree, the branch in its own).
4. Full measured function set by name: 283 → 286, three additions
   (assignRhsValue, clauseHeadEnd, permitRescueMod) and zero removals,
   so nothing was orphaned out of the set.

gofmt -l . lists only paren_singleton_def_test.go, which it already listed on
origin/main; both files this branch touches are clean.

Not fixed (3 shapes MRI refuses and we still accept)

• not a = foo bar — not's operand is an expr, and a = foo bar is
  command_asgn, reachable from stmt only. Needs the assignment RHS to lose
  its command right under not; a narrower question than this change.
• begin; x; else; foo bar; end — an else with no rescue. A different rule
  (bodystmt), unrelated to arg_value.
• f(&b, ) — a trailing comma after a block-pass argument. opt_block_arg is
  last in call_args and admits no comma after it.

🤖 Generated with Claude Code